Apparatus for correcting assembly deviation of an apparatus and correcting a process error using an apriltag, and an apparatus for correcting an assembly deviation of the apparatus and correcting a process error using the same

ABSTRACT

An apparatus for correcting a process error includes: a frame; a machining unit formed inside or outside the frame with respect to the frame and performing a predetermined process; a conveying unit formed inside or outside the frame with respect to the frame and performing predetermined conveying; a sensing mark formed on the frame, the machining unit, or the conveying unit; an imaging unit formed inside or outside the frame and creating an original image by imaging the sensing mark; and a measuring unit deriving a 3D position variation value of the frame, the machining unit, or the conveying unit by deriving an image variation value of the sensing mark by analyzing the original image transmitted from the imaging unit imaging the sensing mark formed on the frame, the machining unit, or the conveying unit.

BACKGROUND OF THE INVENTION Field of the Invention

The present disclosure relates to an apparatus for correcting anassembly deviation and a process error thereof using a sensing mark, anda method of correcting an assembly deviation and process error using theapparatus. More specifically, the present disclosure relates to atechnology of reducing in real time deviations that are caused by gapsand tolerances when an apparatus is assembled and driven by continuouslymonitoring in real time position variation values and angle variationvalues Ox, Oy, and Oz in X, Y, and Z axes of each of a frame, amachining unit, a conveying unit, an actuating unit, a measuring unit, aflexure, etc. that constitute the apparatus using sensing marks, such asan AprilTag, and a camera.

Description of the Related Art

Attempts to 3-dimensionally restore objects have been continuously madethroughout the industry, and it is actually possible to 3-dimensionallyrestore objects with the development of computer vision technology. Whena laser or pattern light is used in such attempts, there is a defectthat the equipment is expensive and difficult to actually use while theaccuracy is high. However, 3D restoration technologies that do not usean artificial light source have an advantage that the equipment issimple although the precision is low in comparison to an active type.

A method of using a camera of such attempts is actively studied due toimprovement of the resolution and performance of cameras. This methodmay be classified into types such as Structure From Motion (SFM), stereovision, and space carving that defines a space with voxels, projects thevoxels to image, and keeps only voxels that satisfy consistency andvisibility of colors. However, these types have a defect that they aredifficult to apply when the restoration target has an insufficienttexture or colors are almost similar.

On the other hand, an AprilTag, which is a visual reference that isuseful in various types of work including augmented reality, robotics,and camera correction, makes it possible to easily create a target incommon printers and to calculate accurate 3D position, direction, etc.of the tag through a camera using sensing software even if there islimitation in lighting or viewing angle.

An AprilTag is conceptually similar to a QR code in that it is a kind of2D barcodes but is designed to encode less data payload (4-12 bits), soit is stronger and can be sensed in a longer range, whereby it ispossible to calculate a 3D position with high accuracy in terms ofdetection rate and accuracy.

A localization method that is performed by a computer system has beendisclosed in Korean Patent Application Publication No. 10-2021-0057586(title of disclosure: Method and system for camera-based visuallocalization using blind watermarking). The computer system includes atleast one processor configured to execute computer-readable instructionsincluded in a memory, and the localization method includes: recognizinga combined image including an invisible marker from a query image; andcalculating a pose of the query image on the basis of the identificationtag of the marker and matched coordinates through the at least oneprocessor.

CITATION LIST Patent Literature

-   Patent Literature 1: Korean Patent Application Publication No.    10-2021-0057586

SUMMARY OF THE INVENTION

In order to solve the problems described above, an objective of thepresent disclosure is to minimize in real time deviations that arecaused by gaps and tolerances when an apparatus is assembled and drivenby continuously monitoring in real time position variation values andangle variation values θx, θy, and θz in X, Y, and Z axes of each of aframe, a machining unit, a conveying unit, a actuating unit, a measuringunit, a flexure, etc. that constitute the apparatus using sensing marks,such as an AprilTag, and a camera.

The objectives to be implemented in the present disclosure are notlimited to the technical problems described above, and other objectsthat are not stated herein will be clearly understood by those skilledin the art from the following specifications.

In order the achieve the objectives described above, the presentdisclosure includes: a frame; a machining unit formed inside or outsidethe frame with respect to the frame and performing a predeterminedprocess; a conveying unit formed inside or outside the frame withrespect to the frame and performing predetermined conveying; a sensingmark formed on the frame, the machining unit, or the conveying unit; animaging unit formed inside or outside the frame and creating an originalimage by imaging the sensing mark; and a measuring unit deriving a 3Dposition variation value of the frame, the machining unit, or theconveying unit by deriving an image variation value of the sensing markby analyzing the original image transmitted from the imaging unitimaging the sensing mark formed on the frame, the machining unit, or theconveying unit.

In an embodiment of the present disclosure, the sensing mark may be anAprilTag, an Aruco marker, an ARtag, or an ARToolKit.

In an embodiment of the present disclosure, the measuring unit maytransmit a control signal to the machining unit or the conveying unit sothat a 3D position of the machining unit or the conveying unit iscorrected in accordance with a 3D position variation value of the frame,the machining unit, or the conveying unit.

In an embodiment of the present disclosure, the sensing mark may becarved, printed, or attached.

In an embodiment of the present disclosure, a conveying unit coupled tothe imaging unit and the frame and moving the imaging unit may befurther included.

In an embodiment of the present disclosure, the measuring unit may senselines in an image of the sensing mark in the original image using leastsquare method (LSM) in clusters of similar pixel gradients.

In an embodiment of the present disclosure, the measuring unit mayderive a 3D position variation value of the frame, the machining unit,or the conveying unit having a sensing mark by analyzing 3D inclinationor movement of the sensing mark.

In order to achieve the objectives described above, the presentdisclosure includes: a first step in which an original image of thesensing mark is taken by the imaging unit; a second step in which themeasuring unit derives an image variation value of the sensing mark byanalyzing the original image transmitted from the imaging unit; a thirdstep in which the measuring unit derives a 3D position variation valueof the machining unit; and a fourth step in which the measuring unittransmits a control signal to the machining unit, whereby the 3Dposition of the machining unit is corrected.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is schematic view of an apparatus according to an embodiment ofthe present disclosure;

FIG. 2 is a perspective view of an apparatus according to anotherembodiment of the present disclosure;

FIG. 3 is a schematic diagram of a conveying unit according to anotherembodiment of the present disclosure;

FIGS. 4A-4D and 5A-5D are images of sensing marks according to aplurality of embodiments of the present disclosure; and

FIGS. 6A-6D are images for analysis of a sensing mark according to anembodiment of the present disclosure.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, the present disclosure is described with reference to theaccompanying drawings. However, the present disclosure may be modifiedin various different ways and is not limited to the embodimentsdescribed herein. Further, in the accompanying drawings, componentsirrelevant to the description will be omitted in order to clearlydescribe the present disclosure, and similar reference numerals will beused to describe similar components throughout the specification.

Throughout the specification, when an element is referred to as being“connected with (coupled to, combined with, in contact with)” anotherelement, it may be “directly connected” to the other element and mayalso be “indirectly connected” to the other element with another elementintervening therebetween. Further, unless explicitly describedotherwise, “comprising” any components will be understood to imply theinclusion of other components rather than the exclusion of any othercomponents.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to limit the present disclosureSingular forms are intended to include plural forms unless the contextclearly indicates otherwise. It will be further understood that theterms “comprise” or “have” used in this specification specify thepresence of stated features, numerals, steps, operations, components,parts, or a combination thereof, but do not preclude the presence oraddition of one or more other features, numerals, steps, operations,components, parts, or a combination thereof.

Hereinafter, the present disclosure is described in detail withreference to the accompanying drawings.

FIG. 1 is schematic view of an apparatus according to an embodiment ofthe present disclosure, and FIG. 2 is a perspective view of an apparatusaccording to another embodiment of the present disclosure. FIG. 3 is aschematic diagram of a conveying unit 300 according to anotherembodiment of the present disclosure.

As shown in FIGS. 1 and 2 , an apparatus for correcting a process errorof the present invention includes: a frame 410; a machining unit formedinside or outside the frame 410 with respect to the frame 410 andperforming a predetermined process; a conveying unit formed inside oroutside the frame 410 with respect to the frame 410 and performingpredetermined conveying; a sensing mark 10 formed on the frame 410, themachining unit, or the conveying unit 300; an imaging unit 100 formedinside or outside the frame 410 and creating an original image byimaging the sensing mark 10; and a measuring unit deriving a 3D positionvariation value of the frame 410, the machining unit, or the conveyingunit 300 by deriving an image variation value of the sensing mark 10 byanalyzing the original image transmitted from the imaging unit 100imaging the sensing mark 10 formed on the frame 410, the machining unit,or the conveying unit 300.

The sensing mark 10 installed on the conveying unit 300 may be imagedwith the conveying unit 300 stopped.

The sensing mark 10 may be an AprilTag, an Aruco marker, an ARtag, or anARToolKit. The sensing mark 10 is described as being limited to thetypes described above in an embodiment of the present disclosure, but itis apparent that the sensing mark 10 is not necessarily limited theretoand other 2D barcodes that can perform the same function may be used.

As shown in FIG. 1 , the machining unit may include a machining tool 210that is a tool for machining a machining target 500, a machining base220 supporting the machining target 500, a tool actuator 230 coupled tothe machining tool 210 and 3-dimensionally moving or rotating themachining tool 210, and a base actuator 240 coupled to the machiningbase 220 and 3-dimensionally moving or rotating the machining base 220.

The machining tool 210, the machining base 220, the tool actuator 230,and the base actuator 240 each may have a multi-axis robot or anorthogonal robot, and the machining unit may be partially or entirelyinstalled inside or outside the frame with respect to the frame. Thecase in which the machining unit is formed inside the frame is describedin the present disclosure for the convenience of description. Themachining unit may be a component including all equipment includingactuating equipment, flexure, etc. required for machining.

The sensing mark 10 may be formed on each of the components constitutingthe machining unit, as described above, the frame 410, etc., and thesensing mark 10 may be carved, printed, or attached.

In detail, the sensing mark 10 may be carved on each surface by a laser,etc., or the sensing mark 10 may be printed on each surface.Alternatively, the sensing mark 10 may be installed by carving orprinting a sensing mark 10 on a substrate and attaching the substrate toeach surface.

The conveying unit 300 is coupled to the imaging unit 100 and the frame410 and can move the imaging unit 100. The conveying unit 300 mayinclude a supporting arm 310 coupled to the imaging unit 100 andsupporting the imaging unit 100, a driver 320 coupled to the supportingarm 310 and 3-dimensionally moving or rotating the supporting arm 310,and a support 330 coupled to the driver 320 and supporting the driver320. The driver 320 may move along the support 330 or may rotate aboutthe support 330.

The imaging unit 100 may be not only coupled to the conveying unit 300,as described above, but also formed at each of a plurality of positionswhere the sensing mark 10 can be imaged inside the frame 410. Theinstallation position and number of the imaging unit 100 may depend onthe process.

However, it may be advantageous that the imaging unit 100 is coupled tothe conveying unit 300 in terms of space use and process efficiency, andit is exemplified that the imaging unit 100 is coupled to the conveyingunit 300 in the present disclosure. It is apparent that the fundamentalprinciple is the same even when the imaging unit 100 is installed at aplurality of positions.

The imaging unit 100 includes a position sensor 110, and informationabout position and posture changes of the imaging unit 100 collected bythe position sensor 110 can be transmitted to the measuring unit. Themeasuring unit transmits a control signal to the driver 320 by analyzingthe information about position and posture changes of the imaging unit100 received from the position sensor 110, whereby the position andposture of the imaging unit 100 can be controlled. The informationobtained by the position sensor 110 may be transmitted to the measuringunit in a wired or wireless type.

The imaging unit 100 that images one sensing mark 10 can image aposition change of the sensing mark 10 only if it images the sensingmark 10 while maintaining a reference position, which is the referencefor imaging the sensing mark 10. Accordingly, the position of theimaging unit 100 can be controlled such that the position, posture, andoptical axis of the imaging unit 100 are maintained for one sensing mark10.

Information about the position for imaging each sensing mark 10 may bestored in the measuring unit. When the imaging unit 100 is moved by theconveying unit 300 to image one sensing mark 10, the measuring unit cantransmit a control signal to the conveying unit 300 so that the imagingunit 100 can perform imaging while maintaining the posture and opticalaxis suitable for imaging the sensing mark 10 at the reference positionfor imaging the sensing mark 10.

Such control of the posture and optical axis of the imaging unit 100 maybe performed in the same way even when the imaging unit 100 is coupledto the frame 410. When the imaging unit 100 is fixed to a portion of theframe 410 and images a sensing mark 10, the position of the imaging unit100 can be controlled such that the imaging unit 100 performs imagingwhile maintaining the posture and optical axis suitable for imaging thesensing mark 10 at the reference position for imaging the sensing mark10. In this case, the driver 320 may be coupled between the imaging unit100 and the frame 410 to control the position of the imaging unit 100.

FIGS. 4A-4D and 5A-5D are images of sensing marks 10 according to aplurality of embodiments of the present disclosure. FIGS. 6A-6D areimages for analysis of a sensing mark 10 according to an embodiment ofthe present disclosure.

FIGS. 4A-4D are images of AprilTags of different embodiments. FIG. 5A isan image of an ARToolKit, FIG. 5B is an image of an ARtag, FIG. 5C is animage of an AprilTag, and FIG. 5D is an image of an Aruco marker.

FIGS. 6A-6D are images for analysis of a sensing mark 10 according to anembodiment of the present disclosure. In detail, FIG. 6A is an originalimage taken by the imaging unit 100, FIG. 6B is an image that showssensing of lines in the original image, FIG. 6C is an image that showssensing of all quads, and FIG. 6D is an image that shows extraction of aquad having a valid code system from the image.

As shown in FIG. 6A, the imaging unit 100 creates an original image byimaging the sensing mark 10, and the original image can be transmittedto the measuring unit. As shown in FIG. 6B, the measuring unit can senselines in the image of the sensing mark in the original image using leastsquare method (LSM) in clusters of similar pixel gradients.

Next, as shown in FIG. 6C, quads can be sensed as many as possible inthe gradient directions, and then, as shown in FIG. 6D, a quad having avalid code system can be extracted from the original image. Themeasuring unit obtains the pose of the sensing mark (AprilTag) 10 in theoriginal image using homograph and intrinsic estimation, measures acoordinate change of each of the apexes of the outermost rectangle, andmeasures 3D inclination or movement, thereby being able to derive animage variation value of the sensing mark 10.

Further, it is possible to derive 3D position variation values of theframe 410, the machining unit, and the conveying unit 300 having thesensing mark 10 by analyzing 3D inclination (angle variation values inX, Y, and Z axes, θx, θy, and θz) or 3D position variation values x, y,and z of the sensing mark 10.

When the apparatus of the present disclosure is in operation, anglevariation values θx, θy, and θz in the X, Y, and Z axes or positionvariation values x, y, and z in the X, Y, and Z axes may be generateddue to spacing at joints by changes in tension, compressing, bending,shearing, twisting, etc. and accumulation of vibration in operation atportions of each part of the machining unit or the frame 410. In thiscase, 3D inclination or movement may be generated at the sensing mark10.

The measuring unit can derive 3D coordinate variation values at eachportion of the sensing mark 10 by analyzing 3D inclination or movementof the sensing mark 10 using a predetermined program, and can derive 3Dposition variation values of the frame 410, the machining unit, theconveying unit 300, etc. having the sensing mark 10 using the derived 3Dcoordinate variation values.

Data obtained for each case by performing simulation on the 3Dcoordinate variation values of each portion of the sensing mark 10 dueto 3D inclination or movement of the sensing mark 10 are stored in theprogram. It is possible to derive a 3D coordinate variation value ofeach portion of the sensing mark 10 by comparing a reference image ofthe sensing mark 10 based on the stored data with data about 3Dinclination or movement of the sensing mark 10 in an original image.

The measuring unit can transmit a control signal to the machining unitor the conveying unit 300 so that the 3D position of the machining unitor the conveying unit 300 is corrected in accordance with the 3Dposition variation value of the frame 410, the machining unit, or theconveying unit 300.

In this case, the imaging unit 100 can continuously image the sensingmark 10, and original images can be transmitted to the measuring unit.When an original image and the reference image of a correspondingsensing mark 10 are the same, the measuring unit can determine that themachining unit has been moved or rotated to a predetermined position andcan finish controlling the machining unit. Control of the machining unitmay include control of each of the machining tool 210, the machiningbase 220, the tool actuator 230, and the base actuator 240. Thisconfiguration can be applied in the same way to control of the conveyingunit 300.

As a detailed embodiment, as shown in FIG. 2 , one module may be formedby forming the machining unit in a housing 420. A plurality of suchmodules may be formed and each of them may be coupled to the frame 410.In this configuration, the imaging unit 100 can image the machining unitformed in each of the modules, and accordingly, a control signal istransmitted to the plurality of machining units from the measuring unit,whereby each of the machining unit can be controlled.

Further, as shown in FIG. 3 , a linear motor 341 and a rotary motor 342may be provided as a configuration corresponding to the driver 320. Anend of the supporting arm 310 may be coupled to the linear motor 341 andthe other end of the supporting arm 310 may be coupled to the imagingunit 100. Further, the imaging unit 100 itself may tilt, rotate, etc. acamera provided in the imaging unit 100, and the imaging range of thecamera may be increased.

Further, the conveying unit 300 may include a motor support 350vertically elongated inside the frame 410 and supporting the linearmotor 341 and the rotary motor 342 may be coupled to the bottom of themotor support 350, so the rotary motor 342 rotates the motor support350, whereby the imaging unit 100 can be rotated.

As described above, when the measuring unit receives information fromthe position sensor 110 of the imaging unit 100, the measuring unittransmits a control signal to one or more of the linear motor 341, therotary motor 342, and a servo motor, thereby being able to control 3Dmovement and rotation of the imaging unit 100. Further, the measuringunit transmits a control signal to the conveying unit 300 to change theimaging position from one sensing mark 10 to another sensing mark 10,thereby being able to change the position of the imaging unit 100.

Further, the sensing mark 10 may be formed on fixed components such asthe motor support 350 and the rotary motor 342, it is possible to imagea sensing mark 10 of the fixed components through other imaging units100 installed on the frame 410, and the measuring unit can measure 3Dposition or rotation variation values of each of the fixed componentsusing original images obtained in this case.

Further, the frame 410 may be included in the fixed component, and theimaging unit 100 images the sensing mark 10 formed on each portion ofthe frame 410 and the measuring unit analyzes the image, whereby it ispossible to measure deformation (3D position or twist variation value)of a portion of the frame 410.

Further, when the measuring unit determines that a fixed component hasbeen 3-dimensionally moved, rotated, or deformed, the measuring unit cantransmit information of the determination to an electronic device of auser and the user can correct wrong position and posture of the fixedcomponent. Accordingly, it is possible to quickly take measures againstproblems with each of the components in the apparatus.

Hereafter, a method of correcting an assembly deviation and processerror using the apparatus of the present disclosure is described.

In a first step, an original image of a sensing mark 10 can be taken bythe imaging unit 100. Further, in a second step, the measuring unit canderive an image variation value of the sensing mark 10 by analyzing theoriginal image transmitted from the imaging unit 100.

Next, in a third step, the measuring unit can derive a 3D positionvariation value of the machining unit. Thereafter, in a fourth step, themeasuring unit transmits a control signal to the machining unit, wherebythe 3D position of the machining unit can be corrected.

The other details of the method of correcting an assembly deviation anda process error according to the present disclosure are the same asthose of the apparatus of the present disclosure described above.

It is possible to minimize a process error due to each component bycorrecting the 3D position value of each component by reflectingdeviations that are caused by gaps and tolerances when an apparatus isassembled and driven by continuously monitoring in real time positionvariation values and angle variation values θx, θy, and θz in X, Y, andZ axes of each of a frame, a machining unit, a conveying unit, anactuating unit, a measuring unit, a flexure, etc. that constitute anapparatus, by using the apparatus and method of the present disclosuredescribed above. That is, it is possible to improve the quality ofproducts that are manufactured by the apparatus of the presentdisclosure by minimizing an assembly deviation of equipment, which isassembled by the apparatus of the present disclosure, a process error inanother process, etc. Further, it is possible to prevent generation ofaccumulated tolerances by minimizing an error that may be generated inan assembly process of equipment (machining unit, etc.) itself installedin the apparatus of the present disclosure.

The present disclosure having the configuration described above has aneffect that it is possible to minimize a process error due to eachcomponent by correcting the 3D position value of each component byreflecting deviations that are caused by gaps and tolerances when anapparatus is assembled and driven by continuously monitoring in realtime position variation values and angle variation values θx, θy, and θzin X, Y, and Z axes of each of a frame, a machining unit, a conveyingunit, an actuating unit, a measuring unit, a flexure, etc. thatconstitute an apparatus.

The effects of the present disclosure are not limited thereto, and itshould be understood that the effects include all effects that can beinferred from the configuration of the present disclosure described inthe following specification or claims.

The above description is provided as an exemplary embodiment of thepresent disclosure, and it should be understood that the presentdisclosure may be easily modified in other various ways without changingthe spirit or the necessary features of the present disclosure by thoseskilled in the art. Therefore, the embodiments described above are onlyexamples and should not be construed as being limitative in allrespects. For example, the components described as a single part may bedivided and the components described as separate parts may beintegrated.

The scope of the present disclosure is defined by the following claims,and all of changes and modifications obtained from the meaning and rangeof claims and equivalent concepts should be construed as being includedin the scope of the present disclosure.

What is claimed is:
 1. An apparatus for correcting an assembly deviationand a process error thereof using a sensing mark, the apparatuscomprising: a frame; a machining unit formed inside or outside the framewith respect to the frame and performing a predetermined process; aconveying unit formed inside or outside the frame with respect to theframe and performing predetermined conveying; a sensing mark formed onthe frame, the machining unit, or the conveying unit; an imaging unitformed inside or outside the frame and creating an original image byimaging the sensing mark; and a measuring unit deriving a 3D positionvariation value of the frame, the machining unit, or the conveying unitby deriving an image variation value of the sensing mark by analyzingthe original image transmitted from the imaging unit imaging the sensingmark formed on the frame, the machining unit, or the conveying unit. 2.The apparatus of claim 1, wherein the sensing mark is an AprilTag, anAruco marker, an ARtag, or an ARToolKit.
 3. The apparatus of claim 1,wherein the measuring unit transmits a control signal to the machiningunit or the conveying unit so that a 3D position of the machining unitor the conveying unit is corrected in accordance with a 3D positionvariation value of the frame, the machining unit, or the conveying unit.4. The apparatus of claim 1, wherein the sensing mark is carved,printed, or attached.
 5. The apparatus of claim 1, wherein the conveyingunit is coupled to the imaging unit and the frame and moves the imagingunit.
 6. The apparatus of claim 1, wherein the measuring unit senseslines in an image of the sensing mark in the original image using leastsquare method (LSM) in clusters of similar pixel gradients.
 7. Theapparatus of claim 1, wherein the measuring unit derives a 3D positionvariation value of the frame, the machining unit, or the conveying unithaving a sensing mark by analyzing 3D inclination or movement of thesensing unit.
 8. A method of correcting an assembly deviation and aprocess error using the apparatus for correcting an assembly deviationand a process error thereof using a sensing mark of claim 1, the methodcomprising: a first step in which an original image of the sensing markis taken by the imaging unit; a second step in which the measuring unitderives an image variation value of the sensing mark by analyzing theoriginal image transmitted from the imaging unit; a third step in whichthe measuring unit derives a 3D position variation value of themachining unit; and a fourth step in which the measuring unit transmitsa control signal to the machining unit, whereby the 3D position of themachining unit is corrected.